1. Field of the Invention
The present invention relates to a device for predicting a frame from other frames, as well as to video coding and decoding devices using the same. More particularly, the present invention relates to an interframe prediction processor, a video coding device, and a video decoding device that perform interframe prediction of video frames on a block basis.
2. Description of the Related Art
Digital video compression techniques are widely used in many applications. MPEG and H.264 are among the standard specifications in this technical field, where MPEG stands for “Moving Picture Expert Group.” The coding and decoding algorithms used in those standards divide each given picture into small areas and process them with motion compensation techniques. Such picture areas are called “macroblocks.” A video coding process involves intraframe prediction and interframe prediction. The intraframe prediction reduces redundancy within a single frame by using orthogonal transform, quantization, and other data compression algorithms. The interframe prediction, on the other hand, reduces redundancy between successive frames by extracting and encoding motion compensation residual (i.e., the difference between a current frame and a motion-compensated reference frame). The resulting video data is then entropy-coded for transmission or storage. A video decoding process reverses the above steps to reconstruct original video from compressed video.
Some standard coding algorithms including MPEG and H.264 adaptively select a frame-based coding mode or a field-based coding mode to process interlaced video signals. These algorithms further permit switching between interframe prediction and interfield prediction for each two vertically adjacent macroblocks (or “macroblock pair”) for motion vector calculation. This coding mode is called “macroblock adaptive field/frame” (MBAFF) mode. Which of those three modes to use can be specified by a “coding type” parameter on an individual picture basis. In this description, the three coding modes will be referred to as “frame,” “field,” and “MBAFF” modes. The first two modes may also be referred to collectively as non-MBAFF mode, where appropriate.
FIG. 56 explains the order of macroblocks to be processed for each different coding type. The topmost part of FIG. 56 shows in what order the coding process selects 16×16-pixel macroblocks constituting a picture. Specifically, the top-left part of FIG. 56 depicts the case of “frame” coding type, in which mode the macroblocks of 16×16 pixels constituting a frame are processed from left to right and top to bottom. This means that odd-numbered lines and even-numbered are selected alternately for processing.
The top-middle part of FIG. 56 shows the case of “field” coding type, in which mode the video data is processed as two separate fields, i.e., top field and bottom field. The coding process begins with a top field (or odd-numbered lines) and then proceeds to a bottom field (or even-numbered lines) in the subsequent field synchronization period, as shown in the bottom-middle part of FIG. 56.
The top-right corner of FIG. 56 shows the case of “MBAFF” coding type, in which a frame is processed on a macroblock pair basis. The processing order of field lines depends on whether the macroblock pair of interest is to be interframe coded or interfield coded. In the case of interframe-coded macroblock pairs (hereafter “frame pairs”), odd-numbered lines and even-numbered lines are processed alternately as illustrated in the bottom-right part of FIG. 56. In the case of interfield-coded macroblock pairs (hereafter “field pairs”), odd-numbered lines are processed before even-numbered lines since the first and second macroblocks of a pair correspond to top field data and bottom field data, respectively.
To improve the accuracy of motion vector prediction for macroblocks containing edges and outlines, some existing coding algorithms split up such macroblocks into smaller blocks and calculate a motion vector for each block. For example, H.264 supports macroblock partitioning that divides a basic macroblock of with a size of 16×16 pixels into various block sizes as necessary, including minimum sub-partitions of 4×4 pixels.
Most part of the video coding algorithm described above is devoted to calculation of motion vectors for interframe prediction. It is therefore desired to develop a faster and more efficient method of motion vector calculation. One such method is proposed in, for example, Japanese Unexamined Patent Application Publication No. 2004-266731. The proposed video coding method alleviates memory capacity requirements of reference pictures by selectively storing a limited number of reference macroblocks that have been determined to be used in the next motion-compensated inter-coded frame.
A video coding device calculates a motion vector (abbreviated as “MV” where appropriate) corresponding to each individual macroblock and then determines a motion vector predictor (abbreviated as “MVP” where appropriate) for a current macroblock from previously calculated motion vectors of its surrounding macroblocks. The video coding device then encodes motion vector differences (abbreviated as “MVD” where appropriate) between MVs and MVPs and outputs them, together with macroblock information, as a coded video data stream. This video stream is received by a video decoding device. The video decoding device decodes the coded MVDs and macroblock information and calculates motion vectors from MVDs and MVPs, where MVP of a macroblock can be determined from motion vectors of surrounding blocks that have already been decoded. Using those motion vectors, the video decoding device reconstructs the original video stream.
A video decoding device has a memory for vector storage locations reserved to store motion vectors of already decoded macroblocks for later reference in reconstructing motion vectors of subsequent macroblocks. More specifically, those vector storage locations accommodate motion vectors of the currently processed macroblock, as well as motion vectors that have been calculated for macroblocks adjacent to the current macroblock.
In MBAFF mode, the MVP of a macroblock is determined from the current macroblock pair and its adjacent macroblock pairs. Since those macroblock pairs may not always be of the same type, the conventional MBAFF algorithm compensates for this inconsistency when reading or writing motion vectors from/to vector storage locations. Suppose, for example, that the current macroblock pair is in frame form while macroblock pairs surrounding it are in field form. In this case, the conventional algorithm converts those adjacent macroblock pairs from field form to frame form before writing motion vectors in the memory. This conversion task imposes an additional processing burden, thus degrading the efficiency of video coders and decoders.